Adaptive Message Pipelining for Network Memory and Network Storage

Recent advances in cluster le systems, network memory, and network-attached disks make it possible to construct storage systems whose performance tracks network technology rather than disk technology. However, delivering the potential of high-speed networks for network storage systems depends on communication support that meets client demands for low network latency for random block accesses and high bandwidth for streaming block accesses. This paper describes an adaptive message pipelining technique that reduces network latency of messages larger than 1 KB while delivering the full bandwidth of gigabit-per-second networks and PCI I/O buses. We report on experiences with the Trapeze messaging system, which implements adaptive message pipelining in rmware for a Myrinet network interface. We show that the Trapeze approach is superior to other message pipelining alternatives, and illustrate its beneets with application experiments using the Global Memory Service, a network memory system on a Myrinet/Alpha cluster. Message pipelining reduces GMS/Trapeze 8KB page fault latency by 38% to 165s, and yields le read bandwidths up to 97 MB/s.